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Abstract 

Students are accessing graduate study online in ever-increasing numbers with interactive 
experiences differing from those who traditionally enroll in corresponding face-to-face (F2F) 
classes. Soft skills such as collaboration/teamwork, communication and presentation are 
important to learning but difficult to practice outside the F2F environment. Robotic telepresence 
units (robots) might benefit distance learners by enriching their online experience, making it 
more similar to corresponding F2F classes. This study examines students’ sense of social 
presence in courses that are accessed fully online, attended via robots or face-to-face. These 
experiences are compared through the lens of Social Presence Theory (Garrison, Anderson and 
Archer, 2000). Of the 227 enrollments in 30 educational leadership courses offered during the 
summer and fall semesters of 2016, a sample consisting of 66 students was selected to measure 
graduate students’ perceptions of social presence. A one-way ANOVA yielded no significant 
differences between groups with regard to overall social presence and when social presence’s 
effect was subsequently compared using eta-squared, only a small effect was found. When asked 
specifically, however, students using robots report an increased sense of class membership; a 
heightened appreciation for peers’ humor, and a greater ability to fonn distinct impressions of 
fellow classmates. These preliminary results give reason to continue pursuit of this line of 
inquiry as subsequent findings could yield important implications for distance education 


programs. 
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Background 

Graduate study via distance education has become fairly common. In 2014, one-third of 
graduate students participated in distance education (NCES, 2016). Private for-profit institutions 
dominate the market but public non-profit schools are serving a substantial portion of graduate 
students; many of the courses in both sectors are offered online rather than in a face-to-face 
(F2F) setting. In 2014, 28% of students enrolled in a public degree-granting institution took a 
distance education course while 17% of this group experienced graduate study exclusively 
through distance education (NCES, 2016). Moreover, it is estimated that between 2014 and 
2025, overall graduate enrollment in distance education will increase by 21% (NCES, 2016). 

Nationally, the growth potential for school leader positions is promising. Educational, 
leadership positions are expected to grow by 6% through 2025 (U.S. Department of Labor, 

2015). A master’s degree is typically required for entry into the field. Hence, it’s reasonable to 
believe that many aspiring school leaders seeking credentialing will choose to avail themselves 
to any one of the hundreds of education leadership graduate programs available online. Such 
courses employ a variety of technologies to deliver instruction either synchronously or 
asynchronously. 

Achieving the promise of equal educational opportunity in Montana includes closing the 
achievement gap for American Indian students (OPI, 2013). Educational leadership plays an 
important role in student achievement (Marks & Printy, 2003). Access to quality educational 
leadership via distance learning is particularly important in rural Montana, a state with a 
population density of only 6.86 people per square mile of land (MTGov, 2016). 
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Theoretical Framework 

Much of learning to lead is essentially heuristic but many agree that a theoretical 
foundation and grounding in the major competencies of the field helps to ensure successful 
future perfonnance (Doh, 2003). In comparing F2F instruction to distance learning the research 
seems to indicate equal learning outcomes (Arbaugh et al., 2013; Lyke & Frank, 2012; Weber & 
Lennon, 2007). Such studies mainly focus on “hard skills” such as budgeting, understanding of 
theory, law, etc. However, leadership is not merely mastery of hard content. It includes “soft 
skills” such as collaboration/teamwork, communication and presentation skills; skills that are 
difficult, if not impossible, to practice in isolation online (Crosbie, 2005). Rather leadership is 
an interactional experience. Classroom-based leadership instruction and simulations serve as a 
constitutive learning experience (Figueroa, 2014). Such experiences are very difficult to 
replicate in an asynchronous setting. 

Robotic telepresence may have the potential to compliment online learning to improve 
the acquisition and development of soft leadership skills or, at the very least, make for a more 
robust distance learning experience for prospective school leaders. A telepresence robot is 
simply a computer-driven robot consisting of a video camera, screen and microphone. In this 
case it is an iPad on a device resembling a Segway (Figure 1). 
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Figure 1. Telepresence Robot. © 2016 Double Robotics, Inc. 

The robot is connected to the internet using Wi-Fi making it possible for the user to 
control it remotely. The remote user (graduate student) can move around the classroom, turn to 
speak to classmates and the instructor, rise up and drop low. In short, it allows remote graduate 
students a means to access face-to-face classes more authentically thus enabling real-time 
synchronous experiences in leadership classes; in particular teaching presence. Synchronous 
instruction establishes a climate that promotes meaningful discourse among students (Szeto, 
2014). 

Review of Literature 

Social Presence Theory (SPT) is helpful for framing the acquisition of “soft skills’ in 
educational leadership courses because it offers a means to better understand graduate student 
experiences between the three instructional milieus; F2F, asynchronous online and the 
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synchronous experience provided by a telepresence robot. Social presence theory was originally 
defined as, “the degree of salience of the other person in the interacting and the consequent 
salience of the interpersonal relationships (Short, Williams and Christie, 1976). For purposes of 
this study we have selected Garrison and Anderson’s (2003) definition, “the ability of 
participants in a community of inquiry to project themselves socially and emotionally, as real 
people through the medium of communication being used.” Garrison & Arbaugh (2007) 
suggested that as students become familiar with each other as the course progresses, social 
presence becomes “transparent.” This study examines the potential for telepresence robots to 
enable the transparency of social presence that Garrison et al. (2007) posited so that distance 
students can engage seamlessly with F2F students. 


Hypothesis and Methods 

Do graduate students accessing educational leadership coursework in a synchronous 
milieu using robotic telepresence report a greater sense of social presence compared to students 
who access coursework in an asynchronous setting? Do social presence perceptions differ 
between face-to-face students and students who access coursework in an asynchronous setting? 
Are social presence scores between students accessing educational leadership coursework in a 
synchronous milieu using robotic telepresence report and those attending class face-to-face 
different? 

To test these questions, we used a one-way between groups analysis of variance 
(ANOVA) and hypothesized that the mean social presence score for at least one of the three 
groups (robotic telepresence, online and face-to-face) would not be statistically equal. Stated in 
the null, we hypothesize that mean social presence scores for graduate students accessing 
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educational leadership coursework would be equal for all three groups hence, Ho: p robotic telepresence 
= p asynchronous online = p face-to-face and we set a priori criteria for rejecting the null at a = .05. 

Missing data were handled by pairwise deletion. 

Significant findings were accompanied by post-hoc analyses to include estimates of 
effect size, specifically eta squared and omega squared. Eta squared the more common statistic 
is prone to overestimating explained variance - particularly with smaller samples (Peters & Van 
Voorhis, 1940; Olejnik & Algina, 2003). To provide a more complete picture we included 
omega squared because, compared to eta squared, it tends to be a less biased effect size estimator 
- especially when small samples are concerned (Carroll & Nordhohn, 1975). 

For the purpose of this study we measured social presence using a survey adapted from 
Lin’s (2004) Social Presence Questionnaire. The survey (see Appendix A) consists of 15 
questions that asked students to rate on a 7-point Likert Scale on how they perceived group 
membership, social comfort and social navigation. Using Qualtrics 1 survey software, a link for 
the survey was embedded in a learning management system webpage or emailed for 14 graduate- 
level courses in educational leadership held during the summer and fall of 2016. The “Prevent 
Ballot Box Stuffing” option in Qualtrics software was set to prevent respondents from taking the 
survey multiple times by placing a cookie on their web browsers. 

Respondents were graduate students enrolled in educational leadership courses at a 
medium size public university in western United States. All survey item responses were recoded 
from lowest to highest to provide consistent logic; high scores represent greater social presence 


1 The data for this paper were generated using Qualtrics software, copyright © 2015 Qualtrics. Qualtrics and all 
other Qualtrics product or service names are registered trademarks or trademarks of Qualtrics, Provo, UT, USA. 
http://www.qualtrics.com 
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than lower scores. Responses were then averaged to obtain a mean social presence score for 
each respondent. 

Results 

Of the 227 enrollments in 30 educational leadership courses offered during the summer 
and fall semesters of 2016, 84 students responded to the survey. Eighteen were excluded from 
the sample (three refused to complete the survey, five were fall respondents who indicated that 
they may have completed the survey during the preceding summer session and ten were deleted 
due to extensive missing data) leaving a sample of 66 graduate students. An examination of all 
remaining missing data did not reveal any patterns. 

Questions #4 through #18 are items that are derived from Lin’s (2004) social presence 
questionnaire. These 7-point questions are scaled from 1= strongly agree to 7= strongly or 
extremely disagree. To make results logical these data were recoded so that high scale values a 
reflect positive opinion while low values correspond to a negative view. Descriptive statistics 
are contained in Table 1. 

To assess internal consistency a Cronbach’s alpha coefficient was computed (a = .93) for 
questions #4 through #18. Based upon Bland and Altman’s (1997) recommended guideline for 
interpretation of a, we conclude that these items cohere quite well and seem to consistently 
measure social presence. 

Based upon Questions #2 and #3 each respondent was categorized as belonging to one of 
three groups: telepresence robot, online or F2F (see Table 2). A mean social presence score for 
each respondent was computed for questions #4 through #18. A one-way ANOVA compared the 
effect of instructional milieu on mean social presence scores for students in online, face-to-face, 
and robotic instructional c. There was no significant difference between the three conditions, 

F{ 2, 63) = .88, p = .42, q 2 = -03 (see Table 3). 
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Within the survey two individual questions, however, yielded significant differences. Again, 
a one-way ANOVA compared instructional milieu in Question #4, “I felt like I was a member of 
the class during the semester.” There was a statistically significant difference between the three 
instructional settings, F( 2, 63) = 3.63 ,p = .03, r| 2 = -10, w p 2 = -07 (see Table 4). Post-hoc 
analyses using Tukey’s HSD indicated that feeling like a member of the class was lowest among 
online students and highest for face-to-race (p = .03) but face-to-face and robotic instruction did 
not differ significantly (p = .28). Based upon the general rule of thumb for interpreting effect 
size 2 using eta and omega squared (Cohen, 1988, Field, 2013), we find a medium effect between 
online and face-to-face milieus for a fonning a sense of class membership. 

Finally, a one-way ANOVA compared instructional milieu in Question #12, “I was able to 
for distinct individual impressions of other students during class activities.” There was a 
significant difference between the three conditions, F( 2, 63) = 8.81,/? = .002, r) 2 = .18, u) p 2 = .15 
(see Table 5). Post-hoc analyses using Tukey’s HSD indicated that fonning distinct individual 
impressions of other students during class activities is lowest among online students and highest 
for face-to-race (p = .01). Additionally, online and robotic instruction differed significantly (p = 
.01) while face-to-face and robotic instruction did not (p = .82). The effect size here is large. 

Discussion 

Let us return to our initial research questions. First, do graduate students accessing 
educational leadership coursework in a synchronous milieu using robotic telepresence report a 
greater sense of social presence compared to students who access coursework in an 
asynchronous setting? Here we were unable to find a difference between synchronous (F2F and 
robots) and asynchronous (online) with respect to social presence. 


2 small = 0.01, medium = 0.06, large = 0.14 
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With respect to overall social presence perceptions differing between F2F students and 
students using a robotic telepresence, again we find no difference. Interestingly we did find a 
significant difference between online and F2F with regard to having a sense of class membership 
with no difference between F2F and robots on the same question. Hence, we infer that there is 
reason to believe that a feeling of belonging is more likely when graduate students attend class 
F2F or with robots but not so when they are solely online. 

Finally, we inquired as to whether social presence scores among students accessing 
educational leadership coursework in a synchronous milieu using robotic telepresence report and 
those attending class F2F differ. Here we find no difference with respect to social presence. 
There was, however, a significant difference between instructional milieus on a question that 
measured a student’s ability to form distinct individual impressions of classmates. In fact, the 
effect was large and no difference between F2F and robotic telepresence students. Again, we 
surmise that F2F and robots are superior to online instructional delivery when it comes to 
gaining familiarity with one’s classmates. 

The small sample size is undoubtedly a limitation to this study and despite non¬ 
significant findings on all three research questions; significant differences concerning developing 
a sense of belonging and increasing familiarity with classmates gives us reason to continue our 
pursuit of this line of research in the context of training school leaders. 
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Tables 
Table 1 

Descriptive Statistics for Selected Survey Questions 


# 

n 

M 

SD 

4 

66 

6.14 

1.14 

5 

66 

6.38 

0.89 

6 

66 

5.05 

1.45 

7 

66 

5.47 

1.64 

8 

66 

5.73 

1.49 

9 

66 

5.89 

1.13 

10 

65 

5.69 

1.25 

11 

65 

5.72 

1.47 

12 

66 

5.39 

1.23 

13 

66 

6.05 

1.12 

14 

66 

5.97 

0.82 

15 

65 

4.83 

1.46 

16 

66 

5.45 

1.38 

17 

66 

5.68 

1.18 

18 

66 

6.23 

1.09 

19 

49 

5.88 

1.54 


Table 2 


Instructional Milieu and Social Presence Scores 


Group 

n 

M 

SD 

Online 

35 

5.59 

0.91 

Face-to-face 

14 

5.9 

0.68 

Robot 

17 

5.83 

0.82 

Total 

66 

5.71 

0.85 


Table 3 

One-Way Analysis of Variance of Mean Social Presence Score by Instructional Milieu 

df SS MS F p 


Between Groups 

2 

Ml 

0.63 

Within Groups 

63 

45.35 

0.72 

Total 

65 

46.61 



0.88 


0.42 
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Table 4 

One- Way Analysis of Variance of Question number four by Instructional Milieu 



df 

SS 

MS 

F 

p n 2 w P 2 

Between Groups 

2 

8.66 

4.33 

3.63 

0.03 0.1 0.07 

Within Groups 

63 

75.11 

1.19 



Total 

65 

83.77 




Table 5 

One- Way Analysis of Variance of Question #12 by Instructional Milieu 

df SS MS F p n 2 Wp 2 

Between Groups 

2 

17.62 

8.81 

6.92 

0.002 0.18 0.15 

Within Groups 

63 

80.14 

1.27 



Total 

65 

97.76 





Table 6 

Mean and Standard Deviation for Survey Question #19 

n M SD 

Robots enhance face-to-face . 00 . 

1 / j.oo 1 

courses 


Note. Scores ranged from 1 (Strongly disagree) to 7 (Strongly agree). 
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Appendix 

Perceptions of Social Presence Survey 

Q1 

O I have read the above information and agree to participate in this research. (1) 

O I do not wish to participate in this research (2) 

If I do not wish to participate in this research is Selected, Then Skip to End of Survey 


Q2 In the past 12 months, have you used one of the ED Leadership Department's robot to 
participate in a face-to-face course? 

O Yes (1) 

O No (2) 

If Yes Is Selected, Then Skip to Q4 I felt like I was a member of the class during the semester 


Q3 In the past 12 months, have you taken more ED Leadership Department classes online 
using UM's Moodle learning management system or face-to-face classes on the campus of the 
UM in Missoula? 

O I have taken more classes online using Moodle (1) 

O I have taken more face-to-face classes on the UM campus in Missoula (2) 

Q4 I felt like I was a member of the class during the semester. 

O Strongly agree (1) 

O Agree (2) 

O Somewhat agree (3) 

O Neither agree nor disagree (4) 

O Somewhat disagree (5) 

O Disagree (6) 

O Strongly disagree (7) 

Q5 I felt comfortable participating in learning activities during the semester. 

O Extremely comfortable (1) 

O Moderately comfortable (2) 

O Slightly comfortable (3) 

O Neither comfortable nor uncomfortable (4) 

O Slightly uncomfortable (5) 

O Moderately uncomfortable (6) 

O Extremely uncomfortable (7) 
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Q6 I felt that I came to know the other students in the class. 

O Strongly agree (1) 

O Agree (2) 

O Somewhat agree (3) 

O Neither agree nor disagree (4) 

O Somewhat disagree (5) 

O Disagree (6) 

O Strongly disagree (7) 

Q7 The course activities helped me to complete assignments with higher quality than if I were 
working alone. 

O Strongly agree (1) 

O Agree (2) 

O Somewhat agree (3) 

O Neither agree nor disagree (4) 

O Somewhat disagree (5) 

O Disagree (6) 

O Strongly disagree (7) 

Q8 The course activities helped me to learn more efficiently than if I were working alone. 

O Strongly agree (1) 

O Agree (2) 

O Somewhat agree (3) 

O Neither agree nor disagree (4) 

O Somewhat disagree (5) 

O Disagree (6) 

O Strongly disagree (7) 

Q9 I felt comfortable addressing my feelings during the course. 

O Extremely comfortable (1) 

O Moderately comfortable (2) 

O Slightly comfortable (3) 

O Neither comfortable nor uncomfortable (4) 

O Slightly uncomfortable (5) 

O Moderately uncomfortable (6) 

O Extremely uncomfortable (7) 
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Q10 I felt comfortable expressing my humor during the course. 

O Extremely comfortable (1) 

O Moderately comfortable (2) 

O Slightly comfortable (3) 

O Neither comfortable nor uncomfortable (4) 

O Slightly uncomfortable (5) 

O Moderately uncomfortable (6) 

O Extremely uncomfortable (7) 

Q11 I was able to appreciate the humor of other students in the course. 

O Strongly agree (1) 

O Agree (2) 

O Somewhat agree (3) 

O Neither agree nor disagree (4) 

O Somewhat disagree (5) 

O Disagree (6) 

O Strongly disagree (7) 

Q12 I was able to form distinct individual impressions of other students during class activities. 
O Strongly agree (1) 

O Agree (2) 

O Somewhat agree (3) 

O Neither agree nor disagree (4) 

O Somewhat disagree (5) 

O Disagree (6) 

O Strongly disagree (7) 

Q13 I was able to learn from the discussions. 

O Strongly agree (1) 

O Agree (2) 

O Somewhat agree (3) 

O Neither agree nor disagree (4) 

O Somewhat disagree (5) 

O Disagree (6) 

O Strongly disagree (7) 

Q14 I learned to value other points of view during the semester. 

O Strongly agree (1) 

O Agree (2) 

O Somewhat agree (3) 

O Neither agree nor disagree (4) 

O Somewhat disagree (5) 

O Disagree (6) 

O Strongly disagree (7) 
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Q15 Actions by other students usually influence me to do additional work. 

O Strongly agree (1) 

O Agree (2) 

O Somewhat agree (3) 

O Neither agree nor disagree (4) 

O Somewhat disagree (5) 

O Disagree (6) 

O Strongly disagree (7) 

Q16 Knowing that other students were aware of my work influences the frequency and/or quality 
of my work. 

O Strongly agree (1) 

O Agree (2) 

O Somewhat agree (3) 

O Neither agree nor disagree (4) 

O Somewhat disagree (5) 

O Disagree (6) 

O Strongly disagree (7) 

Q17 Knowing what other students did helped me know what to do. 

O Strongly agree (1) 

O Agree (2) 

O Somewhat agree (3) 

O Neither agree nor disagree (4) 

O Somewhat disagree (5) 

O Disagree (6) 

O Strongly disagree (7) 

Q18 The courses were a useful learning experience. 

O Extremely useful (1) 

O Moderately useful (2) 

O Slightly useful (3) 

O Neither useful nor useless (4) 

O Slightly useless (5) 

O Moderately useless (6) 

O Extremely useless (7) 
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Answer If Q2 Have you used a robot to participate in an EDLD course? Yes Is Selected 
Q19 Robots enhance face-to-face courses. 

O Strongly agree (1) 

O Agree (2) 

O Somewhat agree (3) 

O Neither agree nor disagree (4) 

O Somewhat disagree (5) 

O Disagree (6) 

O Strongly disagree (7) 



